The Self-organizing Map

information from multidimensional primary signals, and to represent it as a location, say, in a two-dimensional network. Although this i s already a step towards generalization and symbolism, it must be admitted that the extraction of features from geometrically or physically relatable data elements i s still a very concrete task, in principle at least. Theoperation of the brain at the higher levels relies heavily on abstract concepts, symbolism, and language. It is an old notion that the deepest semantic elements of any language should also be physiologically represented in the neural realms. There i s now new physiological evidence for linguistic units being locatable in the human brain [6], [15]. In attempting to devise Neural Network models for linguistic representations, the first difficulty i s encountered when trying to find metric distance relations between symbolic items. Unlike with primary sensory signal patterns for which similarity i s easily derivable from their mutual distances in the vector spaces in which they are represented, it can not be assumed that encodings of symbols in general have any relationship with the observable characteristics of the corresponding items. How could it then be possible to represent the “logical similarity” of pairs of items, and to map such items topographically? The answer lies in the fact that the symbol, during the learning process, i s presented in context, i.e., in conjunction with the encodings of a set of other concurrent items. In linguistic representations context might mean afew adjacentwords. Similarity between items would then be reflected through the similarity of the contexts. Note that for ordered sets of arbitrary encodings, invariant similarity can be expressed, e.g., in terms of the number of items they have in common. O n the other hand, it may be evident that the meaning (semantics) of a symbolic encoding i s only derivable from the conditional probabilities of its occurrences with other encodings, independent of the type of encoding [68]. However, in the learning process, the literal encodings of the symbols must be memorized, too. Let vector x, represent the symbolic expression of an item, and x, the representation of its context. The simplest neural model then assumes that x, and x, are connected to the same neural units, i.e., the representation (pattern) vector x of the item is formed as a concatenation of x, and x,: In other words, the symbol part and the context part form a vectorial sum of two orthogonal components. The core idea underlying symbol maps is that the two parts are weighted properly such that the norm o f the context part predominates over that of the symbol part during the self-organizing process; the topographical mapping 1474 PROCEEDINGS OF THE IEEE, VOL. 78, NO. 9, SEPTEMBER 1990